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Abstract 

In this paper we extend our recent results [P.Jizba, T.Arimitsu Physica A 340 (2004) 110] 
on g-nonextensive statistics with non-Tsallis entropies. In particular, we combine an ax- 
iomatics of Renyi with the g-deformed version of Khinchin axioms to obtain the entropy 
which accounts both for systems with embedded self-similarity and g-nonextensivity. We find 
that this entropy can be uniquely solved in terms of a one-parameter family of information 
measures. The corresponding entropy maximizer is expressible via a special function known 
under the name of the Lambert W-function. We analyze the corresponding "high" and 
"low-temperature" asymptotics and make some remarks on the possible applications. 
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1 Introduction 

The idea that Gibbsian statistical thermodynamics and Shannon's communication the- 
ory share the same line of reasoning was originally introduced by Edwin Jaynes in his 
influential 1957 papers [1] . There he proposed the Maximum Entropy Principle (Max- 
Ent) as a general inference procedure with a direct relevance to statistical physics. A 
standard frame of statistical thermodynamics appeared as soon as the notion of entropy 
was introduced. In particular, Jaynes's MaxEnt utilized Shannon's entropy, or better, 
Shannon's information measure (= S), as an inference functional. The central role of 
Shannon's entropy as a tool for inductive inference (i.e., inference where prior infor- 
mation are given in terms of expectation values) was further demonstrated in works 
of Faddeyev [2], Shore and Johnson [3], Wallis [4] and others. Following Jaynes, one 
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should view the MaxEnt distribution (or maximizer) as a distribution that is maxi- 
mally noncommittal with regard to missing information and that agrees with all what 
is known about prior information, but expresses maximum uncertainty with respect to 
all other matters [1]. 

With the advancement in information theory it has become clear that Shannon's en- 
tropy is not the only feasible information measure. Indeed, many modern communica- 
tion processes, including signals, images and coding systems, often operate in complex 
environments dominated by conditions that do not match the basic tenets of Shan- 
non's communication theory. For instance, buffer memory (or storage capacity) of a 
transmitting channel is often finite, coding can have a non-trivial cost function, codes 
might have variable-length codes, sources and channels may exhibit memory or losses, 
etc. Post-Shannon developments of information theory offer various generalized mea- 
sures of information to deal with such situations. Measures of Havrda-Charvat [5], 
Sharma-Mittal [6], Renyi's [7] and Kapur's [8] can serve as examples. 

If the parallel between information theory and statistical physics has a deeper reason, as 
advocated by Jaynes, then one should expect similar progress also in statistical physics. 
Indeed, in the past 20 years, physicists have begun to challenge the assumptions of 
Gibbs's statistics such as ergodicity or metric transitivity. This happened when evidence 
accumulated showing that there are many situations of practical interest requiring 
statistics which do not conform with Gibbs's exponential maximizers. Examples include 
percolation, cosmic rays, turbulence, granular matter, clustered volatility, etc. 

When trying to generalize Gibbs's entropy, the information-theoretic parallel with sta- 
tistical thermodynamics provides a useful conceptual guide. The natural strategy that 
fits this framework is to revisit the axiomatic rules governing Shannon's entropy and 
potential extensions translate into a language of statistical physical. The usual ax- 
iomatics of Khinchin [9] is prone to several cogent generalizations. Among those, the 
additivity of independent mean information is a natural axiom to attack. In this way, 
two fundamentally distinct generalization schemes have been pursued in the literature; 
one redefining the statistical mean and another generalizing the additivity rule. While 
the first leads to Renyi's entropies [7,10] that are nature tool in systems with embedded 
self-similarity [11], the second scheme yields various deformed entropies [12] that play 
important role in long-range/time correlated systems. 

It is to be expected that a suitable merger of the above generalizations could provide 
a new conceptual frame suitable for a statistical description of systems possessing 
both self-similarity and long-range correlations. Such systems are quite pertinent with 
examples spanning from the early cosmological phase transitions to currently much 
studied quantum phase transitions (frustrated spin systems, Fermi liquids, etc.). Our 
aim was to study one particular merger, namely merger of Renyi and Tsallis-Havrda- 
Charvat (THC) entropies. 

The structure of this paper is the following: In Section II axiomatics of Renyi and THC 



2 



entropies are reviewed. In Section III we formulate a new axiomatics which aims at 
unifying the Renyi and THC entropies. Such an axiomatics allows for only one one- 
parameter family of information measures. Basic properties of this new class of entropies 
are discussed in Section IV. The ensuing maximizer is calculated in Section V. There 
we show that MaxEnt distribution is expressible through the Lambert W-function. We 
analyze the corresponding "high" and "low-temperature" asymptotics and discuss the 
corresponding non-trivial structure of the parameter space. A final discussion is given 
in Section VI. 

2 Renyi's and THC entropies — axiomatic viewpoint 

As already said, RE represents a step towards more realistic situations encountered in 
information theory. Since RE's have a firm operational characterization given in terms 
of block coding and hypotheses testing (see, e.g., [13]), it can be directly measured. 
This is typically happening, e.g., in communication systems with the buffer overflow 
problem or in variable-length coding with an exponential cost constraint. RE's are also 
indispensable in various branches of physics that require self-similar sample spaces. 
Examples being chaotic dynamical systems or multifractals. RE of order q that is as- 
signed to a discrete distribution V = {pi, . . . ,p n } is defined as 



For simplicity's sake we use the base e of natural logarithms. RE thus defined is then 
measured in natural units — nats, rather than bits. 2 

In his original work Renyi [10] introduced a one-parameter family of information mea- 
sures (=RE) which he based on axiomatic considerations. His axioms have been further 
sharpened by Darotzy [14] and others [15]. It has been recently shown in Ref. [11] that 
RE can be conveniently characterized by the following set of axioms: 

(1) For a given integer n and given V = {p u p 2 , . . . ,p n } (Pk > 0, Y2Pk = 1), Z{P) is 
a continuous with respect to all its arguments. 

(2) For a given integer n, X(pi,p 2 , . . . ,p n ) takes its largest value for p k — 1/n (k — 
1,2, ... ,n) with the normalization X ( |, |J = In 2. 



(3) For a given q E R; X(A n B) = 1(A) + X(B\A) with 

1(B\A) = g- 1 (E fe Q k {q)g{X{B\A = A k ))), and Qk (q) = p q J J2kPl withp, = V(A k ). 



(4) g is invertible and positive in [0, oo). 

(5) X{pi,p 2 , . . . ,p n , 0) = X(p 1 ,p 2 , . . . ,p n ), i.e., adding an event of probability zero 
(impossible event) we do not gain any new information. 

These axioms markedly differ from those utilized in [10,14,15]. Important distinction is 
the emergence of the zooming (or escort) distribution g(q) in axiom 3. Note also that 

2 To convert, note that 1 bit = 0.693 nats. 




(1) 
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RE of two independent experiments is additive. In fact, it was proved in Ref. [10] that 
RE is the most general information measure compatible with additivity of independent 
information and the Kolmogorov system of probability. 

Among variety of deformed entropies the currently popular one is the g-deformed Shan- 
non's entropy, better known as THC entropy. As the classical additivity of independent 
information is not valid there, one may infer that the typical playground for THC en- 
tropy should be in systems with non- vanishing long-range/time correlations: e.g., in 
statistical systems with quantum non-locality or in various option-price models. In the 
case of discrete distributions V = {pi, . . . ,p n } THC entropy takes the form: 



S q {V) 



1 



E(p*) 9 -i 



q > 0. 



(2) 



Axiomatic treatment was recently proposed in Ref. [16] and it consists of four axioms 

(1) For a given integer n and given V = [pi,pi, ■ ■ ■ ,p n } (Pk > 0, E£ Pk = 1), S(V) is 
a continuous with respect to all its arguments. 

(2) For a given integer n, S(V) takes its largest value for pk = 1/n (k = 1,2, ... ,n). 

(3) For a given q G K; S(A n B) = S{A) + S(B\A) + (1 - q)S(A)S(B\A) with 
S(B\A) = J2 k Qk{q) S{B\A = A k ). 

(4) S(p 1 ,p 2 ,...,Pn,0) =S(p 1 ,p 2 ,...,p n )- 

As said before, one keeps here the linear mean but generalizes the additivity law. In 
fact, the additivity law in axiom 3 is nothing but the Jackson sum of the q calculus. 



3 Axiomatic merger 



As a natural axiomatic merger of previous two axiomatics one can choose: 

(1) For a given integer n and given V = {pi,P2, ■ ■ ■ ,Pn} (Pk > 0, J2kPk — 1), ^(V) is 
a continuous with respect to all its arguments. 

(2) For a given integer n, T>(V) takes its largest value for pk = 1/n (k = 1,2, ... ,n). 

(3) For a given q G R; V(A n B) = V(A) + V{B\A) + (1 - q)V(A)V(B\A) with 
V{B\A) = r 1 (E fc Qk(q) f (V(B\A = A k ))). 

(4) / is invertible and positive in [0, oo). 

(5) V(p 1 ,p 2 ,...,p n ,0) =V(p 1 ,p2,...,p n )- 

In Refs. [17,18] it has been shown that the above axioms allow for only one one- 
parameter class of solutions given by 

V q (A) = [e-^-i)Y. k e k {i)^Pk _ ^ = _J_ | JJ(p fc )-(i-9)<? fc (5) _ ! j . ( 3 ) 

Here (. . ) q is defined with respect to the distribution Qk(q)- We can further recast the 
relation (3) into another, more convenient, form: 
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V q (A) 



l-q 



1 




(4) 



Eqs.(3)-(4) represent the sought information measure. 
4 Basic properties of V q 

Before studying the implications of the formulas (3)-(4), there is one immediate con- 
sequence which warrants special mention. In particular, from the condition dX q /dq < 
(see, e.g., [7]) one has 



with equality, iff q = 1 or dX q /dq = 0. This happens only when V is uniform or trivial, 
i.e., {1, 0, . . . , 0}. By utilizing the known properties of T q and S q we have 



< S(V) < l q (V) < S q (V) < V q (V) < ln q n, for < q < 1 , 



This means that by investigating the information measure V q with the given q < 1 we 
receive more information than restricting to X q or S q only. On the other hand, when 
q > 1 then both X q and S q are more informative than T> q . In practice one usually 
requires more than one q to gain more complete information about a system. In fact, 
when entropies X q or S q are used, it is necessary to know them for all q in order to 
obtain a full information on a given statistical system [11]. For applications in strange 
attractors the reader may see Ref. [19], for reconstruction theorems see, e.g., [7,11]. 

Let us state here some of the basic characteristics of V q . Among properties that are 
common to both Renyi's and THC entropies we find 



(a) P,(P = {1,0,...,0}) = 

(b) V q {V) > 

(c) V q is decisive, i.e., V q (0, 1) = V q (l,0) 

(d) V q is expansible, i.e., £> ? (pi, . . . ,p n ) = V q (0,p 1 , . . . ,p n ) 

(e) V 1 =X 1 = S 1 = S 

(f) V q involves a single free parameter - q 

(g) V q is symmetric, i.e., P,(pi, ...,p n )= V q (p k{1) ,. . . ,p k(n) ) 

(h) V q is bounded 

Among features inherited from Renyi's entropy we can find 



> S q (A) if q < 1 
< S q (A) if q > 1 



(5) 



< Vg(V) ^ Sg(V) ^ Xg(V) K S (V) < 111 W, 



for q > 1 . 



(6) 
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(i) V q (A) = r l (Ek Q k (q)f(V q (A k ))) 

(j) V q is a strictly decreasing function of q, i.e., dV q /dq < 0, for any q > 

Result (i) follows from the fact that T> q is a monotonically decreasing function of 
(In V) q and that (In 7-% is a monotonically increasing function of q. Finally, properties 
imprinted from Tsallis entropy include 

(k) ma^ v V q (V) =V q (V = {1/n, . . . , 1/n}) = \n q n 

(1) V q is q non-extensive, i.e., V(A n B) = V{A) + V(B\A) + (1 - q)V(A)V(B\A) 

The issue of thermodynamic stability will be discussed separately in Section 5.1. 
5 MaxEnt distributions for V q 

According to information theory, the MaxEnt principle yields distributions which re- 
flect least bias and maximum ignorance about information not provided to a recipient 
(or observer). Important feature of the usual Gibbsian MaxEnt formalism is that max- 
imizers are all grater than zero and that the maximal entropy is a concave function of 
the values of the prescribed constraints [20] . 

Let us first address the issue of the T> q maximizer. We start by seeking the conditional 
extremum of V q subject to the constraints imposed by the g-averaged value of energy E: 

(E) q = J2Mq)E k . (7) 

k 

By considering the normalization condition for pi we should extremize the functional 

L q (V) = V q (V)-Q^^-^ Pk , (8) 

with Q and $ being the Lagrange multipliers. Setting the derivatives of L q (V) with 
respect to pi, . . . ,p n to zero, we obtain 

- n q (E i -{E) q )^—-^ = 0, i = 1,2,. ..,n. (9) 

Note that when q — > 1 then (9) reduces to the usual condition for Shannon's maximizer. 
This, in turn, ensures that in the q — > 1 limit the maximizer boils down to Gibbs's 
distribution. To proceed we note that Eq.(9) can be cast to the form 

S(Pi) 1_, £(P*) 9 = e^ hlV ^(q((\nV}q-\n Pl )-l)-qn(E t -(E}q). (10) 
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By multiplying both sides by Qi(q), summing over % and taking the normalization con- 
dition J2kPk = 1, we obtain 



$ = _ e (?-i)(inP> 9 



ln(-$) 
9-1 



max — 



g-1 



+ (ii) 



Plugging result (11) back into (10) we have 



E(p*) 9 = (p0 9_1 
k 



qlnpt+ U-^-f {Ei - { E) q ) 



which must hold for any index i. On the substitution 



g - 1 



$ -9-°*' 



we finally obtain the equation 



(12) 



(13) 



(14) 



This has the solution 



Pi = 



1 w ( ~ ^ e (i-mh 



11/(1-9) 



K(q-l) 



w f K{q-1) e { q -l)Si/q\ 
' \ (,,■■!) 



(15) 



with W(x) being the Lambert W^-function [21]. 

Some comments are now in order. First, piS as prescribed by (15) are positive for any 
value of q > 0. This is a straightforward consequence of the following two identities [21]: 



W(x) = £ 



71=1 



_^n-l n n-2 

(n- 1)! 



W(a;) 



x e 



-W(x) 



(16) 



Indeed, the first relation ensures that for x < also < and hence W(x)/x > 0. 

Thus for < q < 1 the positivity of pj's is proven. Positivity for q > 1 follows directly 
from the second relation. Second, as q — > 1 the entropy T> q ^ S and hence pj's defined 
by (15) approaches the Gibbs distribution. To see that this is the case, let us realize that 



$L=i = -1, £L=i = l + H + n(Ei-{E)), and k| 



q=l 



= 1. 



(17) 



Then 
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Pi\ q =i= exp(l- (l+H + n(Ei- (E)))) = exp(QF-Q^) = e" niSi /Z, (18) 

which after identification £l\ q=1 = (3 leads to the desired result. Note also that (15) 
is invariant under uniform translation of the energy spectrum, i.e., the corresponding 
Pi is insensitive to the choice of the ground-state energy. Third, there does not seem 
to be any simple method for determining $ and fl in terms of (E) q . In fact, only 
asymptotic situations for large and vanishingly small Q can be successfully tackled. 
For this purpose we briefly remark on the asymptotic behavior of Pi in regard to Q. 

We first assume that fl <C 1 — " high-temperature expansion" — then from (16) follows 




< l - g) M-m 



' -exp ( (l-q)^A q Ei 



W(x)[l - (l-q)n*A q Ei] , 



n* 



$(W(x) + 1) ' 
The relation (15) then implies that 



x = 



K(q-l) fq-V 
exp 



Pi 



[1 - (l-q)n*A q E t 



|V(l-9) 



E fe [l - {l-q)n*A g E k 



|V(9-1) 



= Z-'ll (l-g)fi*A^] 1/(1 ^, (19) 



with the partition function 



k 



q)Q*A q E k 



lV(l-9) 



/c(g-l) 



W(a:) 



V(9-l) 



(20) 



The distribution (19) agrees with the so called 3rd version of thermostatics introduced 
by Tsallis et al. [22] . It can by also formally identified with the maximizer for RE [23] . 
Clearly, f2* is not a Lagrange multiplier, but £1* passes to (3 at q — > 1 (in fact, $ — * — 1, 
f2 — > /3 and W(x) — > at q — > 1). Note also that when = (i.e., no energy constraint) 
then pi — l/n which reconfirms that T> q reaches its maximum for uniform distribution. 

From the physical standpoint it is the asymptotic behavior at Q\(q — 1)/<&| 3> 1 - 
" low-temperature" expansion — that is most intriguing. This is because the branching 
properties of the Lambert W-function at negative argument make the structure of V 
non-trivial. To this end one can distinguish four distinct situations: 



a{) (q — 1) > and 
6i) (q - 1) < and 



A q Ei < , 
A q Ei < , 



a 2 ) (q- 1) > and A q Ei > 0, 
62) (g- 1) < and A q Ei > 0. 



Cases a±) and 02) are much simpler to start with as the argument of W is positive. W 
is then a real and single valued function which belongs to the principal branch of W 
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known as W . When A q Ei < then a±) implies W(z) z and hence 

Pi = (J^ A V^exp^A^ = Z^exp^-^A^ , (21) 

with 

/ I \ 1/(5-1) 

Zl - (w) el/ '- 

Note that in this case pj is of a Boltzmann type. On the other hand, the a 2 ) situation 
implies the asymptotic expansion [21]: 

W(z) « ln(z) - ln(ln(z)) Pl = Z^ 1 [1 - (1 - g)lT A^] 1/(1 ~ 9) , (22) 



with 



Zo = 



K{q-l) 



In 



«(g-l) 



11/(5-1) 



IT 



1*1 



Although the distribution (22) formally agrees with Tsallis et al. distribution, it cannot 
be identified with it as Q* does not tend to f3 in the q — > 1 limit. In fact, the limit 
q — > 1 is prohibited as it violates the "low-temperature" condition Q\(q — 3> 1. 

Note particularly that our MaxEnt distribution represents in the "low-temperature" 
regime a heavy tailed distribution with Boltzmannian outset — behavior typical, e.g., 
for income distributions. When Q and q > 1 are fixed one may find k and $ from the 
normalization condition and sewing condition at A q E = 0. However, because the "low- 
temperature" approximation does not allow to probe regions with small A g E one must 
numerically optimize the sewing by interpolating the forbidden parts of A q E axis [18] . 

Cases bi) and 62) have much richer structure than ai) and a 2 ). This is due to the 
negativity of the argument that enters the W function. A remarkable upshot of this is 
an existence of a strongly suppressive effect in the occupation of the high-energy states. 
In addition, the suppression appears in two different ways depending on the value of 
(1 — g)/|$|. Analogous type of behavior is know in quantum phase transitions [24]. 
Complete discussion of this phenomenon will be presented in Ref. [18]. 



5.1 Thermodynamic stability — concavity issue 



In the following we are going to address the issue of thermodynamic stability. Note 
that in contrast to information-theoretic entropy T> q , "P^max is the system entropy, 
i.e., it depends on the system state variables. Thermodynamic stability then consists 
of showing that 2^1 max is a concave function of the energy constraint [20]. So we wish 
to show that 
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<9 2 XVP)| m ax &V q ((E) q ) 
This can be done by observing that [21] 



dW{x) W(x) d 2 W{x) W{x) 2 {W{x) + 2) 

rfx ~ + 1) an rfx 2 1 x 2 {W{x) + l) 3 ' 

If we combine (24) with the fact that dV q (V)\ max /d& — l/(q — 1) we obtain 



(25) 

Thus is thermodynamically stably for any q. 



6 Conclusions 



In this paper we have reviewed the main aspects of the recently proposed information 
measure T> q . In contrast to presently popular generalizations based on deformed en- 
tropies, we have aimed here at a strictly axiomatic approach. This is because we hold 
that one cannot proceed the formal generalization of the entropy in physics by ignoring 
the consistency with information theory. As a rule, axiomatic treatments of informa- 
tion measures have the benefit of a closer passage to operational characterizations and 
hence to a systematic use in practical applications. 

We hope that the proposed axiomatics might serve as a novel playground for q- 
nonextensive systems with embedded self-similarity. Indeed, our conclusions hint that 
V q could play a relevant role in quantum phase transitions and/or in econophysics. 

The reader may note that we have not checked T> q for Lesche's observability cri- 
terium [25] (also known as experimental robustness). This is because in our view the use 
of Lesche's condition as the stability criterion is rather doubtful, see e.g., Refs. [26,27]. 
In this connection Yamano's local stability criterion [27] would seem more appropriate 
concept to use. Work along those lines is currently in progress. 

Finally we should stress that the presented entropy V q has many desirable attributes: 
like THC entropy it satisfies the nonextensive g-additivity, involves a single parameter 
q, goes over into S in the limit q — > 1, it complies with thermodynamic stability, conti- 
nuity, symmetry, expansivity, decisivity, etc.. On that basis it would appear that both 
S q and V q have an equal right to serve as a generalization of statistical thermodynamics. 
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